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Abstract 

It is well known that if a submartingale X is bounded then the increasing pre- 
dictable process Y and the martingale M from the Doob decomposition X = Y+M 
can be unbounded. In this paper for some classes of increasing convex functions / 
we will find the upper bounds for lim n sup x Ef(Y n ), where the supremum is taken 
over all submartingales (X n ),0 < X n < l,n = 0,1,.... We apply the stochastic 
control theory to prove these results. 
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1. Introduction. Let (Q, J 7 , (J r n )n>o, P) be a filtered probability space, X = 
(X„) n > be a bounded submartingale on it, < X n < 1, and let 

X n = Y n + M n ,n = 0,1,... (1) 

be its Doob decomposition, where Y = (Y n ) n > is a predictable nondecreasing random 
sequence, 

n 

Y n = E(X k - X fc _i|7- fc _i), n > 1, 
k=i 

and M = (M„) n > is a martingale. Denote by G(0, 0) the class of all such submartingales 
with X = Y = 0, defined on a finite, n = 0,1,..., AT or an infinite time interval, 
n — 0, 1, ....Though X is bounded, F and M can be unbounded, respectively from above 
and from below. Probably, the simplest example of such kind is when X n takes only two 
values, and 1, and 

P(X n+1 = l\X n = 0) = P(X n+1 = 0\X n = 0) = 1/2, 

P(X n+1 = l|X n ) = l) = l. 

Then P(F n+1 - Y n = l/2\X n = 0) = 1 and P(F n+1 - Y n = 0\X n = 1) = 1. If X = F = 
M = 0, then we have the Doob decomposition (1) with 

1 1 n_1 
^= 2 + 2^ 1 (1 " Xfe) ' n = 1 ' 2 '-- 

The transition probabilities defined above imply 

P(X n = 0) = (l/2)",P(X n = 1) = 1 - (1/2)™, 

P{Y n = fc/2) = P(M n = -fc/2) = (l/2) fe , 1 < fc < n, 

P(Y n = n/2) = P(M n = -n/2) = (1/2)"" 1 . 

Now, one can check that lim n Ef(Y n ) < oo if /(x) = a;™ for any m > or if f(x) = 
exp (\x) with A < 2 In 2 ps 1.386, and this limit is infinite for A > 2 In 2. 
This example raises a natural question about the values (estimates) of 

c n = c n (/)= sup f?/(y n ), c(/) = limc n (/) (2) 
xeG(o,o) n 
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for different functions /. The partial answers for these questions were given in [1], where 
in particular it was proved that for function f(x) = x m for any m > 1 the estimate for 
the upper bound is m m . The case of exponential / was not analyzed. 

The main result of this note is Theorem 1 and the inequality (6), which we call the 
shift inequality. 

Theorem 1. 

(a) For functions f(x) = e Xx , A > 0, 

c(f) = 1/(1 -A), if < A < 1, 
= oo, if 1 < A. 

(b) For functions f(x) = x m ,m > 1, c(f) < m m . 

(c) For any increasing convex function f(x),x > 0, with concave derivative f'(x), 
c(f) < B < oo, where B is the unique solution of the equation 

B = f(o) + nr i (B)). (3) 

Remark 1. The results from [2] imply that c(f) is finite for all increasing convex 
functions with derivative of the form 

f(x) = f(0)exp{[ X \(s)ds}, (4) 
Jo 

where A(s) > and lim s _^oo A(s) = Ao < 1, but the expression for B is more complicated. 

We will show also that the structure of submartingales (X k ), where the upper bounds 
are achieved, have the structure similar to the example above, i.e. Xk takes only two 
values, and 1, and AYk + ± = Yfc+i — Yk = a n -k, where the constant a n -k depends only 
on remaining time {n — k). 

Before proving Theorem 1, we present briefly the main steps of the proof grouping 
them into three parts A, B and C. 

A. To obtain the estimates for c(f) we consider a problem of stochastic control 
on the time interval [0,n], where the control actions are the choices of the increments 
AY k +i = Y k+ i — Y k and AM k+ i = M k+1 — M k for k — 0, 1, ...,n — 1, and the goal is to 
maximize Ef(Y n ), where / is a convex increasing function. Later n tends to infinity. 
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The estimate for this functional gives an estimate for c n (f) and its limit gives the value 
of c(f) in (2). 

B. To obtain the latter statement we show that c(f) is bounded, c(f) < lim n b n , 
where b n is the solution of the following recursive equation 

b n = sup \af\a) + (1 - a)f\a + /-'(Vi))]. (5) 

0<a<l 

The sequence b n is nondecreasing and has a finite limit for some functions / and 
infinite for others. The problem here is to describe the class of functions / with finite 
limit. The equation (5) is of interest on its own, though its interpretation is not quite 
clear. 

C. The reduction of the problem of stochastic control to the recursive equation (5) 
is possible through the use of the following inequality which we call the shift inequality 

Ef(a + Y)<f(a + f- 1 (Ef(Y))). (6) 

Denote S the class of all increasing functions f(x),x > for which (6) holds for any 
number a > and any random variable Y > 0. It is easy to see that S contains all 
exponential functions (with equality in (6)). We will show (Lemma 2) that this class also 
contains all power functions of the form f(x) = x m ,m > 1, and all increasing convex 
functions with concave derivatives. We provide a simple example of convex function for 
which (6) is not true. We fail to find the references in the literature to the inequalities 
of such type. A substantial generalization of inequality (6) was obtained in [2]. 

We describe the problem of stochastic control in Section 2 and prove the shift inequal- 
ity for some functions / in Section 3. The reduction of the stochastic control problem 
to the recursive equation (5) is given in Section 4. Section 5 contains the solution of the 
recursive equation for different classes of functions.. We describe some open problems in 
Section 6. 

We thank C. Striker who draw our attention to paper [1] and him, F. Delbaen, M. 
Emery, and J. Franchi for useful discussion concerning the shift inequality. 

2. A stochastic control problem. 
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Similarly to G(0, 0), denote G(x,y) = {all submartingales X = (X n ) such that 
< X n < 1, X — x, and Y — y, 0<x<l,0<y< oo}, and given a convex increasing 
function /, 

c n (f,x,y)= sup Ef(Y n )= sup Ef(y + Y n ), (7) 

XeG(x,y) XGG(x,0) 

where (Y n ) is a predictable sequence from decomposition (1). The second equality in (7) 
holds because (X' n ) e G(x,0) iff (X n ) e G(x,y), where X n = X' w Y n = Y^ + y,n = 0, 1, ... 

As in many problems of stochastic control where n tends to infinity, it is convenient 
to consider the maximization problem in inverse time. Thus we consider time intervals 
of the form (n, n — 1, 1, 0) and use corresponding notation. 

Formally, we consider a Markov Decision Process (MDP) with states (k,x,y), where 
k = 0,1,..., and < x < 1, < y < oo (see [3]). The set of all actions (controls) 
admissible at the state (k,x,y) is {(a,rj) : a is a number, 0<a<l — x, i] is & random 
variable such that Erj = 0, and 0<x + a + r/<l}. We denote D(x) = {rj : Erj = 0, 
< x + i] < 1}. 

The goal of control is to maximize E njXjy f(Y ) over all possible strategies, where 
a strategy is a sequence of admissible actions, maybe nonstationary and randomized, 
(n, x, y) is an initial state, and Y is a (random) position of a last coordinate at the last 
moment k = 0. The value function in this problem we denote F n (x,y). In notation from 
(2) and (7) c n (f) = F n (0,0), and c(f) = lim n F n (0,0). 

Obviously 

F (x,y) = F n (l,y) = f(y), n = 0, 1, ... (8) 
The Bellman equation for our problem takes a form 

F n (x,y)= sup { sup EFn^x + a + r],y + a)} = supa^Fn^x^), (9) 

0<a<l-2, r/£D(x+a) 

where operators T a g(x, y) = sup^T^g^x, y), T a ' v g(x, y) = Eg[x + a + rj,y + a). 

Before to describe the structure of optimal actions in Lemma 1, we need the following 
simple statement. Let us denote £(x) G D(x) a random variable taking value (1 — x) 
with probability x, and value — x with probability (1 — x). 
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Proposition 1. Let h(s) be a convex function on [0, l],x G [0, 1]. Then 
s up veD(x) Eh(x + r]) = Eh(x + £(x)) = xh(l) + (1 - x)h(0). 

Proof. The set D(x) is convex and closed in a weak topology. The random variable 
£(x) is an unique extreme point and hence Ef(x + rj) reaches its maximum at £(#). 

Lemma 1. For the problem of stochastic control described above on a finite or an 
infinite time interval 

a) there is an optimal action (a n (x,y),r) n (x,y)) at each state (n,x,y), where n = 

0. 1, 2, ... is a remaining time, 

b) the second component of the optimal action has a form i] n (x, y) = £(x + a n (x, y)), 
where the random variable is described above. 

c) F n (x,y) is an increasing function in y given n,x, and a decreasing and convex in 
x ( constant for n = 0) given n, y. 

Proof. Point a) follows from the general theory of MDP Point b) says that X n = 
or 1 for all n, except maybe the initial moment, with probability 1. We will prove points 
b) and c) by induction on n. 

For n = we have Fo(x,y) = f(y), i.e. c) holds. Suppose that Lemma 1 is proved 
for ra — l. Since F n _i(x,y) is convex in x, Proposition 1 immediately implies point b) of 
Lemma 1 for n. Then, using also the second equality in (8) for (ra — l), we obtain that 
in the Bellman equation (9) 

T a F n _i(x, y) = [(x + a)f(y + a) + (1 - (x + a))F n _x(0, y + a)}. (10) 

and hence the Bellman equation takes a form 

F n (x,y)= sup [(x + a)f(y + a) + (l-(x + a))F n _ 1 (0,y + a)]. (11) 

0<a<l-x 

Since F n _i(0,y) and f(y) are increasing in y, formula (11) implies that F n (x,y) is 
also increasing in y. 

To prove the monotonicity property of F n (x, y) in x, let us note that if < x\ < x 2 < 

1, and action (a,rj) is admissible at x 2 then action (a + x 2 — xi,rj) is admissible at x\. 
Then formula (10) implies that T a+X2 ~ xl F n _i(xi, y) = T a F n ^i(x2,y) + (x2 + a)[f(y + a + 
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x 2 — xi)— f(y+a)]+(l — (x 2 +a))[F n _i(0, y+a+X2— xi) — F n _i(0, y+a)]. The monotonicity 
of functions F n _i{x,y) and /(y) in y implies that the latter expression is positive. Since 
this is true for any a admissible at X2, we obtain that F n (xi,y) > F n (x 2 ,y). 
Now let us prove the convexity of F n (x,y) in x, i.e. the inequality 

(F n (x 1 ,y) + F n (x 2 ,y))/2> F n (x,y), x x + x 2 = 2x, < x x < x 2 < 1. (12) 

Let an action a n (x, y) = a is an optimal action at state (n, x, y), and thus < a < 1 — x. 

Suppose first that a = 1—x. Then by Lemma 1 r) n (x,y) = £(1) = and by (11) and 
the second equality in (8) 

F n (x, y) = F n _!(l, y + 1 - x) = /(y + a). (13) 

Let us consider any points Xi,x 2 such that Xi + x 2 = 2x. Denote Oj = 1 — Xj. Then 
a = 1 — x = (ai + a 2 )/2. Using formula (10) for x = Xj, i = 1, 2 we obtain 

F n (xi,y) >T a >F n ^(x t ,y) = F n ^(l,y + a,) = f(y + ai ),i = 1,2. 

By convexity of function / we have (/(y+ai)+/(y+a2))/2 > f(y+a), and therefore, 
using (13), we obtain (12). 

Suppose now that a < 1 — x. In this case we prove (12) for Xi, x 2 , such that x x + x 2 = 
2x and |xj — x|/2 < 1 — (a + x), % — 1,2.. This of course implies the convexity of F n (x). 
Note that for such Xj an action (a, £(xj + a)) is an admissible action. Let us show that 
for such Xi, x 2 

[T^+^^y) +T a '^ 2+a )F„_ 1 (x 2 ,y)]/2 = F n (x,y) = T^^F^x, y). (14) 
By the definition of operators T a,r? we have 

T^{x+a) Fn _^ y ) = ( x + a )f( y + a ) + (i _ ( x + a ))F n _ 1 (0, y + a), 

and 

T^^+^F^^Xi, y) = ( Xi + a)f(y + a) + (1 - (x, + a))F n _i(0, y + a), i = 1, 2. 

Taking the average of the last two equalities and using the equality x\ + x 2 = 2x we 
obtain (14). Since F n (x i) y) > T a,r? F n _i(xj, y) for all admissible (a, 77), then (14) implies 
(12). Lemma 1 is proved. 
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3. The Shift inequality (6). The description of the class S. 

Now we turn our attention to the description of the class S of all increasing functions 
f(x),x > 0, not necessarily convex, for which (6) holds. In [2] it was proved that a 

necessary and sufficient condition for twice continuously differentiable functions / to be 
in S : 4t7^t is a nonincreasing function. 

f'(x) to 

To keep our paper selfcontained we present a brief and different proof for the cases 
covered in our Theorem 1. 

Lemma 2. Class S contains 

a) all exponential functions f(x) = e Xx , A > 0, 

b) all power functions f(x) = x m ,m > 1, and 

c) all increasing convex functions with concave derivatives. 

Proof. The first statement is checked trivially. 

To prove b) and c) note that we can rewrite (6) as 

f-\Ef(a + Y))<a + f-\Ef(y)) (15) 

and one has the equality if a = 0. 

b) Let f(x) = x m , m > 1, and \\X\\ m = (ElX)™) 1 /" 1 . Then by the triangle (Minkovski) 
inequality 

f-\Ef{a + Y)) = \\a + Y\\ m < a + \\Y\\ m = a + f-\Ef(Y)). 

c) The derivative (with respect to a) of the left-hand side term in (15) is equal to 

\f-UEf(a + Y)Y - Ef{a + Y) 

[f {Ef{a + Y)] ~ f'V-HEf (a + ¥)))> 

where in the numerator we use (Ef(a + Y))' = Ef'(a + Y) since / is continuously 
differentiable. Since / is a convex and increasing function, we have Ef(a + Y) > 
f(E(a + Y)), and therefore f~ l {Ef\a + Y)) > E(a + Y), and f'(f-\Ef(a + Y))) > 
f'(E(a + Y)) > Ef'ia + Y), where the last inequality is true since /' is concave. Thus 
we obtained that [f~ 1 (Ef(a + Y)\ < 1 and therefore (15). Lemma 2 is proved. 
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Remark 2. Two last cases may suggest a conjecture that the shift inequality would 
be true for all convex slowly increasing functions. It is easy to see that the inequality 
is wrong for the following slowly increasing function : f(x)—xiiO<x<l,f(x) — 
(1 + :r 2 )/2 if 1 < a; < oo, and a — 1, Y is a symmetrical Bernoulli random variable. 

4. Prom the Bellman equation (9) to the recursive equation (5) through 
the shift inequality (6). 

Our final goal is to estimate c(f) = lim n F n (0, 0). By Lemma 1 and (11) we have 

F n (0, 0) = sup (af(a) + (1 - a)(F n _i(0, a)). (16) 

0<a<l 

By definition F n _i(0, 0) = sup^ Ef(Y ), where ir is an admissible strategy and Y is 
a corresponding position of y. Denote H(n — 1,0,0) the set of all possible Y for an 
initial point (n — 1, 0, 0). The set of all admissible strategies for any initial point (k, x, y) 
does not depend on y. Hence H(n — 1,0, a) = a + H{n — 1,0,0) and F n _i(0, a) = 
su P_ff(n-i,o,a) Ff(Y ) = sup if ( n _ 100 ) Ef(a + Y ). This statement is just a paraphrase of a 
second equality in (7). 

Suppose that a convex function / belongs to the class S, for which the shift inequality 
(6) holds. Then Ef(y + Y ) < f(y + f~ 1 (Ef(Y ))) and therefore, using the fact that 
both functions / and f^ 1 are strictly increasing, we have 

F n _!(0,a) = supEf(a + Y ) <sup f(a + f-\Ef(Y )) = 

TT TT 

= /(a + /"'(sup Ef(Y )) = f(a + f-^F^O, 0)). 

TT 

Combining this with (16) and using notation F n (0, 0) = c n , we obtain the inequality 
c n < sup \af\a) + (1 - a)f(a + f' 1 ^))}. (17) 

0<a<l 

Comparing this sequence with a sequence (b n ) defined by the recursive equation (5), 
and assuming that b = c = /(0), we obtain that c n < b n for all n, and hence the 
upper estimate for lim6 n can serve as an estimate for limc n . Note that for exponential 
functions inequality in (17) become an equality and therefore c n = b n for all n and 
c(f) = lim&„. 
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5. The solution of the recursive equation (5). 

Given a convex increasing function /, let us consider the function g(a,b) = af(a) + 
(1 — a)f(a + and the function G(b) = sup 0<fl<1 g(a, b). Then b n = G(b n -i),n = 

1, 2, bo = /(0). Since g(0, b) = b we have b < G{b) and 6 n _i < b n . 

Denote 

5 = inf{6 > : dg(a, b)/da < for all < a < 1}. (18) 

Since dg(a, B)/da < for all a, and g(0,B)= B we have sup a g(a, B) = B. Function 
g(a, b) is increasing in b for any fixed a. Hence, if b < B then sup a g(a, b) < sup a g(a, B) = 
B. So we immediately obtain the following 

Proposition 2. A sequence b n is increasing and lim n 6 n < B. 

Our goal is to describe functions / for which B < oo and to find conditions when 
lim6 n = B. We have 

dg(a, b)/8a = g'(a, b) = f(a) + af'(a) - f(a + f'^b)) + (1 - a)f'(a + f-\b)). (19) 
In particular g'(0, b) = f(0) -b + /'(/ _1 (6))- 

Proposition 3. If g'(a, b) > c> 0, for all < a < e, e > 0, then G(b) >b + ce. 

Proof. The definition of G(b) and a condition of Proposition 3 imply that G(b) > 
g{z, b) > g(0, b) + ce = b + ce. 

Proposition 3 implies immediately a simple sufficient condition for lim„6„ = oo. 

Proposition 4. If for some e > 0, g'(a, b) > c > 0, for all < a < e and all b > 0, 
then lim n b n = oo. 

Proof of Theorem 1. (a) For function f(x) = e Xx , X > 0, we have 

g'(a, b) = [e Xa {a + (1 - a)b)}' = e Xa {l + 6(A(1 - a) - 1) + Xa) 

and 

^(0,6) = 1 + 6(A-1). 

Therefore: 

10 



if A > 1 then it is easy to see that the condition of Proposition 4 holds for sufficiently 
small e, and we obtain that lim n b n = oo. 

if A < 1 then it is easy to see that B — 1/(1 — A) and if lim n b n = 6* < B we obtain 
a contradiction with Proposition 3. 

If A = 1 then g'(a,b) = e a (l + a(l — b)) and starting with b = /(0) we obtain 
&! = G(bo) = f(l) = e. For b > 2 we have sup a #(a,6) = g(l/(b- 1), 6) = e 1 /( 6_1 )(6 - 1) 
and if lim n 6 n = 6* < oo then 6* must satisfy the equation b — (b — l)e 1 ^ b ~ 1 \ It is easy 
to check that this equation has no solution. Thus lim„6 n = oo. 

(b) Consider the function f(x) = x m ,m > 1. Formula (19) gives 

g'(a, b) = {m + \){a m - a{a + b 1 '™)™- 1 ) + (a + 6 1/m ) m_1 (m - b 1/m ). 

Therefore B = m m and lim n b n = b 1f < B. It is easy to check that if b < B then there are 
c > 0, e > such that g'(a, b) > c > for all < a < e which contradicts to Proposition 
3. Hence 6* = B. 

(c) Formula (19) can be rewritten as 

g'(a, b) = f(a) + a[f(a) - f'(a + /-»)] + [f'(a + f-\b)) - f(a + /-»)] (20) 
= f(a) + ah 1 (a,b) + h 2 (a,b). 

Since /' is increasing we have hi(a,b) < 0. Since / is a convex and increasing and /' 
is a concave then h 2 (a,b) < for sufficiently large b, and tends to — oo as b — > oo. It 
implies that for sufficiently large b the derivative g'(a,b) < for all < a < 1. Hence 
B < oo, where B is defined in (18). As in point (b) one shows that the inequality 
lim„6 n = b* < B implies a contradiction with Proposition 3. Since h' 2 (a,b) < for all 
a, we obtain also that the value of B defined in (18) can be found from the condition 
g'(0, b) = 0. In other words the value of B defined in (18) is also a solution of an equation 
B = f(0) + f'(f-\B)). 

6. Some solved and some open problems. 

The following problem was solved in [2]. 

Problem 1. Describe the class of all increasing functions f(x),x > 0, not necessarily 
convex, for which (6) holds. 
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The following problems are open. 

Problem 2. Find precise estimates for c(f) in cases b) and c) of Theorem 1. 

Problem 3. Obtain results about the possible growth of Ef(Y n ) when instead of 
boundedness of (X n ) some assumptions on its growth are imposed. 

Problem 4. Find an independent and natural interpretation of the recursive equa- 
tion (5). 
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